Pre-trained text encoders have drawn sustaining attention in natural language processing (NLP) and shown their capability in obtaining promising results in different tasks. Recent studies illustrated that external self-supervised signals (or knowledge extracted by unsupervised learning, such as n-grams) are beneficial to provide useful semantic evidence for understanding languages such as Chinese, so as to improve the performance on various downstream tasks accordingly. To further enhance the encoders, in this paper, we propose to pre-train n-gram-enhanced encoders with a large volume of data and advanced techniques for training. Moreover, we try to extend the encoder to different languages as well as different domains, where it is confirmed that the same architecture is applicable to these varying circumstances and new state-of-the-art performance is observed from a long list of NLP tasks across languages and domains.

Intel recently told investors that its 10-nanometer "Cannon Lake" chips won't be ready until 2019, but by then, it might get leapfrogged by its PC arch-rival. AMD told investors that it's sampling next-gen Zen 2 processors with 7-nanometer tech in preparation for a launch in 2019. The company is also testing a Radeon Instinct machine learning graphics cards that use 7-nanometer Vega-based technology. The GPUs will be manufactured by TSMC, said AMD CEO Dr. Lisa Su, but the company will also use Global Foundries, presumably for processors and other products. AMD recently launched its second-generation Zen chips, which use current 14-nanometer 10.